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INTRODUCTION 


With  the  introduction  of  computer  enhanced  telemanipulators  (surgical  robots),  complex  surgical 
procedures  such  as  thoracoscopic  coronary  artery  anastomosis  or  laparoscopic  radical 
prostatectomies  have  become  feasible  (1,2).  Robot  assisted  endoscopic  procedures  are  associated 
with  a  definite  learning  curve  (3),  where  surgical  technical  skills  are  expressed  as  a  function  of 
time  used  for  training.  The  standard  way  of  gaining  the  necessary  surgical  technical  skills  has 
until  recently  been  by  training  in  a  dry  lab  followed  by  training  on  animal  models  (2)  or,  more 
commonly,  by  learning  technical  skills  in  the  operating  room  using  the  100  years  old  Ilalstedian 
principle  “see  one,  do  one,  teach  one”  (4). 

The  possibility  of  using  virtual  reality  simulators  in  surgical  training  was  proposed  more  than  a 
decade  ago  (5).  This  form  of  training  holds  the  potential  of  reducing  the  need  for  mechanical 
models  and  animals  in  surgical  training  without  compromising  surgical  outcome  in  the  operating 
room  (OR).  Mechanical  trainers  are  being  used  to  train  and  evaluate  laparoscopic  surgical  skills 
(6),  but  compared  to  virtual  reality  simulators  the  assembly  of  mechanical  trainers  is  time 
consuming.  After  each  session  mechanical  trainers  have  to  be  reassembled  and  prepared  again 
for  the  next  student,  and  they  do  not  allow  automated  measurements  of  surgical  performance. 
Seymour,  et  al.  have  published  a  study  which  claims  to  show  that  virtual  reality  training 
improves  operating  room  performance  (7). 

SimSurgery  (8)  has  developed  technology  for  virtual  reality  simulation  with  special  focus  on 
surgical  suturing  and  soft  tissue  deformation  such  as  tissue  dissection.  SimSurgery 's  vision  is  to 
increase  clinical  performance  and  to  reduce  healthcare  costs  by  offering  solutions  for  better 
training  and  computer  assisted  tools  in  surgery.  SimSurgery  was  established  in  September  1999 
to  develop  and  sell  medical  simulation  software  for  virtual  training  and  planning.  The  company’s 
unique  software  for  simulating  minimally  invasive  surgery  (MIS)  has  been  developed  by  a 
closely  knit  team  of  highly  competent  specialists.  The  first  product  for  training  laparoscopic 
suturing  sold  in  the  EU  and  US  in  2003. 

Master  slave  robots  used  in  minimally  invasive  surgery  operate  by  transforming  the  surgeon’s 
hand  movement  into  fine  precise  instrument  maneuvers.  The  most  dominating  vendor  of  surgical 
robot  systems  is  Intuitive  Surgical  (Sunnyvale,  CA)  (9)  with  its  master  slave  robot  da  Vinci.  The 
3  or  4  arms  moving  the  surgical  instruments  (slave)  are  controlled  by  a  surgeon  console  (master) 
that  convert  the  movement  of  the  surgeons  hands  into  precise  instrument  trajectories  in  a  very 
elegant  and  intuitive  way.  The  mechanical  system  comprising  the  master  console  is  rather 
complex  (expensive  and  large  size)  to  meet  demands  for  clinical  use. 

However,  it  is  possible  to  develop  a  device  that  utilizes  a  modem  motion  tracking  system  similar 
to  the  surgeon  robot  console.  A  robot  console  developed  only  for  training  is  not  limited  by  the 
clinical  requirements  experienced  in  a  real  robot  system.  Such  a  motion  tracking  based  system 
has  potential  for  being  less  complex  and  less  expensive  than  the  clinical  console. 


BODY 

4.1  Significance  and/or  Uniqueness  of  the  Effort 

To  develop  a  portable  surgical  robot  simulator,  solutions  for  five  fundamental  problems  outlined 
below  must  be  found.  SimSurgery  has  been  addressing  these  problems  with  a  strong  focus  and 
significant  commitment  for  five  years  (company  established  in  1999).  During  these  years,  basic 
research  in  mathematics,  physics  and  computer  graphics  from  Europe  and  the  US  has  been 
utilized.  SimSurgery  has  patents  pending  that  address  solutions  for  surgical  interfacing,  suturing 
and  soft  tissue  modeling.  SimSurgery  is  part  of  a  corporate  structure  with  expertise  in  3D 
animation  and  graphical  entertainment  solutions.  SimSurgery  has  access  to  cutting-edge 
technology  and  competence  within  the  field  of  computer  graphics  and  computer  networks  for 
distributed  solutions.  The  SimSurgery  roots  are  partly  from  the  Interventional  Centre  at  the 
Norwegian  National  hospital.  This  research  centre  for  telemedicine  and  surgical  robotics  was  one 
of  the  first  European  sites  performing  robot  assisted  operations.  The  head  of  the  Interventional 
Centre,  Erik  Fosse,  is  an  experienced  war  surgeon  who  will  participate  in  the  project. 

The  5  fundamental  problems: 

1)  Development  of  a  surgical  interface  that  transforms  the  surgeon’s  instrument  maneuvers  into 
similar  movements  of  the  virtual  reality  instruments. 


SimSurgery  has  developed  a  new  type  of  instrument  interface,  the  SimPack™,  focusing  on 
measuring  the  movement  of  instruments,  whether  they  are  inside  or  outside  the  simulated 
patient’s  body.  This  approach  allows  random  placement  of  ports  relevant  for  different  types  of 
procedures  and  patients  (e.g.  operation  on  children  and  obese  patients).  The  instrument  holder  is 
specially  designed  for  easy  change  of  different  surgical  instrument  grips  such  as  needle  drivers, 
graspers  and  video  scopes.  Just  as  it  is  flexible  in  terms  of  different  grips;  it  can  be  used  in  robot 
assisted  laparoscopy,  robot  assisted  open  surgery  and  non-robotic  open  surgical  procedures. 
SimPack™  is  constructed  for  easy  portability  and  can  be  attached  to  ordinary  laptop  or  desktop 
computers.  SimPack™  will  be  produced  as  a  commercial  product  for  laparoscopic  training  in 
Q4-2004,  but  the  potential  use  for  robotic  and  open  surgery  needs  further  development. 

2)  Solutions  for  modeling  soft  tissue  with  proper  geometric  representation  that  is  undergoing 
transformation  due  to  instrument  interaction,  tissue  cutting  and  removal. 


In  modelling  objects,  SimSurgery  applies  a  core  technology  called  Sim3DM®.  In  simple  terms, 
Sim3DM®  enables  virtual  objects  to  behave  like  real  ones  in  the  medical  field.  Software 
packages  such  as  3D  Studio  Max  can  help  a  user  produce  3D  animation.  Although  the  user 
probably  will  not  notice,  these  standard  software  packages  generally  build  a  3D  object  from  a 
large  number  of  basic  shapes  (often  polygon  based).  When  the  3D  objects  arc  made  to  change 
shape,  rotate  or  otherwise  move  around,  the  computer  (or  graphic  card)  will  recalculate  the 
position  of  all  the  small  geometries  that  make  up  the  object.  For  a  number  of  reasons,  this 
standard  modelling  approach  cannot  easily  be  applied  in  simulated  surgery. 
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•  In  standard  3D  modelling,  rendering  is  often  slow  because  the  movement  of  an  object 
requires  heavy  computation  of  each  building  block  -  simulation  will  not  be  in  real  time. 

•  Standard  3D  modelling  is  slow  at  detecting  whether  one  object  touches  another  object; 
the  process  involves  calculating  the  position  of  every  node  or  triangle  in  an  object  and 
then  checking  if  another  object  occupies  the  same  position. 

•  Standard  3D  modelling  cannot  easily  incorporate  deformable  objects.  (Your  Nintendo 
racing  game  would  typically  show  the  car  either  unscratched  or  wrecked.) 

•  Standard  3D  modelling  is  not  well  adapted  at  giving  measured  responses  when  objects 
interact.  In  a  3D  video  game,  when  one  object  hits  another,  you  would  typically 
experience  just  a  few  possible  outcomes:  the  body  either  stands  or  falls. 


Sim3DM®  has  a  different  approach  compared  to  standard  computer  assisted  animation.  This 
makes  it  ideal  for  highly  demanding  medical  simulation  applications.  Like  a  sculptor  or 
sculptress  working  in  a  stone  quarry,  Sim3DM®  first  cuts  the  basic  building  block(s).  The  block 
is  then  refined  to  a  rough,  square  figure  defined  by  a  mesh.  Then,  the  surfaces  are  “polished”  into 
an  even  shape  by  a  set  of  equations  that  define  the  curves  of  the  object.  This  approach  is  efficient 
inasmuch  as  it  builds  a  3D  model  with  less  data  and  allows  for  efficient  rendering,  collision 
checks  etc.  Each  part  of  the  Sim3DM®  object  can  be  assigned  properties  such  as  elasticity  and 
connectivity  (how  one  part  of  the  object  relates  to  other  parts).  When  exposed  to  a  force 
(pressure,  dragging,  etc.)  the  software  can  easily  calculate  how  the  object  responds. 

3)  Solutions  for  modeling  a  surgical  suture,  stitching  and  interaction  between  several  soft  and 
moving  objects  such  as  the  tissue,  instruments  and  the  suture 


We  model  the  suture  as  a  sequence  of  segments  where  the  length  of  the  total  thread  is  invariant. 
Some  of  the  segments  are  embedded  in  tissue  and  the  rest  are  located  in  free  air.  One  segment 
ends  and  another  starts  where  the  thread  leaves  the  air  and  enters  the  tissue  or  vice  versa.  These 
stitch  points  together  with  the  end  points  of  the  whole  thread  make  the  vertices  of  a  "stitch 
polygon".  When  the  length  of  the  stitch  polygon  is  shorter  than  the  prescribed  fixed  length  of  the 
thread,  one  or  more  of  the  free  segments  have  slack,  otherwise  they  are  straight  lines.  The 
embedded  segments  are  always  represented  as  straight  lines.  The  distribution  of  slack  between 
the  segments  is  done  in  an  ad  hoc  manner.  When  the  stitch  polygon  is  longer  than  the  thread,  the 
stitch  polygon  must  be  replaced  by  a  shorter  one,  meaning  the  tissue  is  pulled  together.  A  blend 
polygon  of  correct  length  can  be  found  by  solving  some  nonlinear  equation  system. 

A  free  segment  of  prescribed  length,  only  constrained  at  its  end  points,  can  be  modelled  as  an 
elastic  curve  with  flexural  stiffness  and  mass  density.  Animation  of  the  segment  is  then  modelled 
as  a  sequence  of  stationary  states  with  minimum  potential  energy  due  to  bending  and  gravity.  A 
stationary  state,  and  hence  the  curve  shape,  is  determined  by  the  prescribed  length  and  spatial 
end  conditions  for  the  segment.  The  bending  energy  of  an  arc  length  parameterized  curve  is 
proportional  to  the  integral  of  its  squared  second  derivative. 

To  avoid  the  free  segment  penetrating  tissue;  tools  and  the  model  for  free  segments  must  be 
modified.  Dynamic  models  make  use  of  hit/collision  detection  and  repulsive  forces  between  the 
thread  and  objects  not  to  be  penetrated. 
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4)  Visual  realism  in  terms  of  graphics. 


SimSurgery  exploits  all  new  technology  offered  by  the  graphic  industry  such  as  graphic  cards 
and  modeling  software  such  as  Maya.  SimSurgery  has  unique  expertise  in  providing  graphical 
realism.  SimSurgery  also  has  a  close  collaboration  with  animation  groups  producing  advance  3D 
animated  motion  pictures  and  clinical  groups  in  Norway.  The  clinical  partners  provide  important 
feedback  in  the  modeling  of  surgical  scenes  and  other  material  such  as  videos  and  access  to 
PACS  archives  for  3D  image  data. 

5)  Realism  in  terms  of  physical  models  controlling  the  soft  tissue  models  (dynamic  behavior  of 
the  geometry-). 

As  for  the  physical  model,  the  finite  element  model  is  a  discrete  formulation  of  a  continuum. 
Another  method  is  the  spring-mass  model  that  represents  a  division  of  the  physical  object  into 
particles  (distributed  masses)  with  springs  connecting  the  particles.  The  spring-mass  model 
yields  a  simpler  and  more  flexible  model,  but  does  not  give  as  realistic  responses  as  finite 
element  models.  SimSurgery  has  used  combinations  of  different  models.  It  should  be  noted  that 
the  Sim3DM  data  structure  allows  easy  integration  of  new  physical  models,  optimized  according 
to  surgical  requirements  and  computer  speed. 

4.2  Hypothesis 

It  is  possible  to  develop  a  portable  device  that  resembles  the  surgeon  console  in  a  surgical  robot 
system,  as  well  as,  surgical  simulator  software  that  replaces  the  need  for  biological  tissue 
(animals  or  patients)  and/or  equipment. 

4.3  Technical  Objective 

The  technical  objective  of  this  project  was  to  develop  a  portable  device  that  resembles  the 
surgeon  console  in  a  surgical  robot  system,  as  well  as,  surgical  simulator  software  that  replaces 
the  need  for  biological  tissue  (animals  or  patients)  and/or  equipment.  The  first  phase  focused  on 
the  development  of  a  small  demonstration  system  for  training  basic  skills  such  as  instrument 
manipulation  and  suturing.  The  next  phase  will  be  to  develop  a  specific  robot  assisted  procedure 
utilizing  the  portable  robot  master. 

4.4  Military  Significance: 

The  proposed  project  has  relevance  for  the  US  military  in  4  ways: 

1)  Use  in  surgical  educational  programs  within  the  US  military  for  learning  robot  assisted 
surgery  or  open  surgical  procedures. 

The  portable  robot  simulator  will  be  designed  with  an  educational  model  that  directly  relates  to 
general  surgery  standards.  This  will  include  an  embedded  performance  measurement  of  terminal 
learning  objectives  for  robotic  surgery  or  open  surgery.  The  educational  model  centers  on  the 
development  of  terminal  learning  objectives  for  surgery  procedures  focusing  on  cognitive, 
psychomotor,  and  judgment  skills.  The  learning  objectives  will  reside  and  be  embedded  as  part 


of  the  portable  platform.  For  example,  based  on  the  terminal  learning  objectives,  it  is  likely  that 
much  of  the  training  objectives  reside  in  the  graphics  and  visualization  component  so  that  the 
robotic  approach  is  transferable  to  general  surgical  training.  This  approach  will  facilitate 
development  of  a  performance  model  that  matches  the  appropriate  technical  features  to  specific 
learning  objectives.  In  addition,  within  the  core  software  architecture,  specific  patients  will  be 
created  for  particular  surgical  scenarios.  These  will  include  robot  assisted  laparoscopy,  robot 
assisted  open  surgery  and  open  surgery  in  general  with  task  components  such  as  debreedement, 
arterial  bleed,  dissection,  spleenectomy,  appendectomy,  and  colon  resection.  Embedded  student 
performance  metrics  will  be  included. 

2)  Use  for  training  paramedics  to  deal  with  trauma  and  open  surgical  procedures  by  providing  a  6 
degree  of  freedom  interactive  environment. 

The  two  handed  6  degree  of  freedom  SimPack™  (with  additional  grip  and  other  instrument 
controls)  enables  interactive  environments  for  hand-eye  coordination  exercises.  In  addition  to 
using  the  platform  in  learning  basic  surgical  tasks  relevant  for  trauma  scenarios,  the  platform 
provide  a  portal  to  train  on  communication  procedures,  triage  and  decision  making  in  combat 
situations.  This  is  accomplished  by  connecting  several  SimPorts™  together  in  a  network 
interfaced  to  the  same  virtual  world.  SimSurgery's  special  access  to  3D  animation  technology 
and  surgical  competence  with  war  surgery  experience  has  been  an  important  part  of  the 
company’s  uniqueness  since  it  was  established. 

3)  With  surgical  robots  at  remote  sites,  the  project  offer  solutions  for  learning  to  use  the  surgical 
robot  and  for  telementoring  surgeons  on  the  remote  site  using  the  robot. 

One  of  the  potential  benefits  of  using  surgical  robots  for  the  U.S.  military  is  the  possibility  of 
using  the  robots  on  remote  sites  with  limited  access  to  surgical  specialists.  The  robot  can  be  used 
as  an  enabling  tool  to  help  the  surgeon  carry  out  complex  operations,  perhaps  with  little  on  site 
assistance.  In  addition  to  being  an  advanced  tool,  the  robot  is  also  a  platform  that  enables  remote 
control  (telesurgery)  or  remote  guidance  (telementoring).  The  portable  robot  simulator  is  a  cost 
effective  device  to  learn  and  it  is  a  practical  and  cost  effective  device  for  telementoring.  A  large 
group  of  experts  having  access  to  the  platform,  offers  the  possibility  of  finding  the  right  expert  to 
assist  in  emergency  situations  at  remote  sites. 

4)  With  future  surgical  robots  that  are  completely  controlled  by  a  remote  center  for  use  in  combat 
fields,  the  project  offers  a  solution  to  learn  the  use  of  such  robots  without  the  cost  of  training  on  a 
real  system. 


There  are  many  ongoing  projects  trying  to  develop  robot  systems  that  can  carry  out  emergency 
tasks  in  combat  fields  or  other  dangerous  environments.  The  use  of  such  robot  systems  requires 
remote  control  of  the  robot  from  dedicated  personnel  with  the  right  skills  and  experience.  A 
portable  simulator  would  be  a  cost  efficient  device  in  training  and  guidance  without 
implementing  such  robot  systems. 
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4.5  Methods 


The  standard  method,  and  so  far  the  only  way  to  train  robot  assisted  surgery,  is  to  practice  with  a 
complete  robotic  setup  including  OR  facilities.  Such  assembly  of  mandatory  equipment  makes 
surgical  robot  training  very  expensive,  thus  render  a  significant  limitation  in  providing  robot 
assisted  procedures  to  suitable  patients.  By  exploiting  surgical  simulator  technology,  it  is 
possible  to  simulate  with  a  PC  anatomical  organs  (the  patient),  the  OR  and  much  of  the  robotic 
equipment.  Since  the  master  slave  robots  used  in  minimally  invasive  surgery  operate  by 
transforming  movements  of  the  surgeons  hand  into  fine  precise  instrument  maneuvers,  it  is 
possible  to  develop  a  device  that  utilizes  modem  motion  tracking  system  in  a  way  similar  to  the 
surgeon  robot  console.  A  robot  console  developed  only  for  training  is  not  limited  by  the  clinical 
requirements  encompassed  in  a  real  robot  system. 

SimSurgery  platform  (SEP)  that  has  been  prepaed  and  tested  with  2  surgical  robot  systems  (da 
Vinci  and  Zeus  from  Intuitive  Surgical,  http://www.intusurg.com).  SEP  is  a  general  platform  for 
computer  based  surgical  training  and  education.  It  is  a  complete  learning  system  including  both 
hardware  and  software  components  that  emulate  a  real  laparoscopic  situation.  SEP  is  unique  in 
design  with  regard  to  how  the  surgeon  interacts  with  the  simulator  (SimPack™)  and  the 
variations  of  training  exercises.  The  surgical  interface  SimPack™  offers  the  possibilities  to 
resemble  different  body  surfaces  with  different  port  placements  and  enables  procedure  specific 
anatomy  exploration.  The  simulator  software  that  uses  a  standard  PC  offers  a  large  range  of  skill 
training.  The  simulator  displays  both  abstract  and  procedure  realistic  scenes.  Many  basic  skills 
can  be  learned  in  an  abstract  environment  and  more  advanced  skills  can  be  learned  in  procedure 
realistic  environments.  Skill  assessment  is  by  validated  metrics  that  reflect  the  user’s  competence 
level.  The  administration  unit  in  SEP  is  easily  combined  with  all  kinds  of  additional  educational 
material  by  embedding  tools  to  access  multimedia  inside  the  software.  SEP  includes  modular 
software  for  easy  update  of  new  training  modules.  The  system  has  individual  applications  for 
administration,  training  in  roaming  mode  (free  selection  of  exercises),  and  training  in  session 
mode  (predefined  curriculum).  In  the  session  mode,  it  is  possible  to  require  that  the  user  exceed  a 
certain  performance  level,  before  he/she  can  continue  to  the  next  set  of  training  exercises. 


KEY  RESEARCH  ACCOMPLISHENTS 


•  First  version  (vl)  of  the  portable  robot  master  console  for  simulation  -  called  SimPack  Robot 
-  has  been  developed  and  functionality  tests  have  been  carried  out.  Successful  testing  has 
initiated  final  design  for  commercial  product. 

•  21  of  25  laparoscopic  simulator  exercises  have  been  converted  to  robot  assisted  exercises. 

•  SimSurgery  personnel  have  evaluated  the  21  working  exercises  in  relation  to  robotics  versus 
traditional  laparoscopy.  The  evaluation  has  concluded  that  the  simulator  software  must 
manage  and  record  instrument  collision.  Instrument  collision  handling  will  be  included  in  the 
commercial  launch  of  the  software. 


REPORTABLE  OUTCOMES 


Publications: 

•  Seymour  NE,  Rjatnes  JS:  Challenges  to  the  development  of  complex  virtual  reality 
surgical  simulations.  Surgical  Endoscopy  2006  (in  press,  November  2006) 

Presentation  of  the  robot  prototype  simulator  at  the  following  exhibits: 

•  SimSurgery  demonstrated  SEP-Robot  in  the  TATRC  exhibit  at  the  MMVR  conference 
(Medicine  Meets  Virtual  Reality)  in  Los  Angeles,  USA,  January  2006. 

•  SEP-SurgicalSim  Education  Platform  was  used  during  training  courses  and  exhibited  at 
SAGES  2006  Annual  Meeting  in  Dallas,  Texas,  USA,  April  2006. 

•  SEP-SurgicalSim  Education  Platform  was  used  during  training  courses  and  exhibited  at 
10th  World  Congress  of  Endoscopic  Surgery  and  14th  International  Congress  of  the 
European  Association  for  Endoscopic  Surgery  (EAES),  Berlin,  Germany,  September 
2006. 

A  commercial  robot  simulator  to  be  launched  at  the  end  of 2006. 

•  News  and  brochures  -  www.simsurgery.com 


CONCLUSION 

The  proposed  project  aimed  to  develop  a  portable  device  that  resembles  the  surgeon  console  in  a 
surgical  robot  system,  as  well  as,  surgical  simulator  software  that  replaces  the  need  for  biological 
tissue  (animals  or  patients)  and  OR  equipment.  The  first  phase  focused  on  the  development  of  a 
small  demonstration  system  for  training  basic  skill  such  as  instrument  manipulation,  tissue 
dissection  and  suturing.  The  next  phase  will  be  to  develop  a  specific  robot  assisted  procedure 
utilizing  the  portable  robot  master.  This  will  address  basic  surgical  training  objectives  also 
transferable  to  open  surgical  procedures. 
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